Caching for flash-based databases and flash-based caching for databases

Database storage systems today are primarily based on two technologies: HDD (hard disk drive) and DRAM (dynamic random-access memory). It is increasingly difficult for these systems to deliver acceptable performance, due to fast expanding data volume, growing energy concern, and cost constraints. The emergence of flash memory has made cost-effective solutions possible. However, conventional storage systems are designed without the knowledge of flash memory limitations and flash device characteristics. Therefore, they can not fully exploit the potential of flash memory. This dissertation investigates two major aspects of flash-incorporated database storage systems. The first aspect is related to the buffer management issues of two-tier storage systems where flash devices are used as the primary storage, i. e., caching for flash-based databases. The second aspect is related to the mid-tier cache management issues for three-tier storage systems where flash memory is used as a page cache to speed up accesses to the slower primary storage, i. e., flash-based caching for databases. The major contributions can be summarized as follows: • It identifies the weaknesses of previously proposed buffer algorithms for flashbased storage systems and, as improvement, proposes the CFDC (clean-first dirty-clustered) algorithm, which is one of the earliest proposals addressing the flash random write problem. • It examines the parameter tuning problem, which discourages the practical use of various previously proposed buffer algorithms, and proposes the CASA (cost-aware self-adaptive) algorithm, which automatically adapts itself to the extent of device R/W asymmetry and to changing workloads at runtime. • From an architectural perspective, it empirically compares conventional storage systems and three-tier storage systems with flash as the mid-tier cache and delivers indicative implications to system designers. • It identifies the cold-page migration problem in flash-based mid-tier caching and proposes two effective solutions. The results suggest an important architectural consideration that was ignored so far—native management of flash memory by the mid-tier cache manager.